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PREFACE 


,  ’‘*S 


This  technical  paper  summarizes  and  briefly  describes  work  accomplished 
under  Project  2362,  Computer  Based  Maintenance  Aids  for  Technicians.  The 
objective  of  the  project  is  to  develop  and  evaluate  a  prototype  computer  based 
system  for  the  storage,  retrieval,  and  presentation  of  technical  data  for  use 
by  maintenance  personnel  to  maintain  Air  Force  Systems.  The  prototype  system 
and  the  knowledge  gained  in  developing  and  evaluating  it  are  to  be  the  basis 
for  the  development  of  an  automated  technical  data  system  for  operational  Air 
Force  use. 

The  work  described  in  this  technical  paper  was  accomplished  under  three 
contracts.  The  first  effort.  Human  Factors  Study  of  an  Automated  Technical 
Order  System,  was  accomplished  by  Behavioral  Technology  Consultants,  Inc. 
under  contract  F33615-77-C-0043.  Dr.  Thomas  W.  Frazier  was  the  principal 
investigator.  The  second  effort  was  accomplished  by  Behavioral  Technology 
Consultants,  Inc.  under  conract  F33615-78-C-0030.  Dr.  Thomas  W.  Frazier  was 
the  principal  investigator  for  this  effort  also.  The  third  effort  discussed 
in  this  paper  was  accomplished  by  Unified  Industries,  Inc.  Mr.  Walter  Holmes 
was  the  principal  investigator.  The  human  factors  portion  of  this  effort  was 
accomplished  under  subcontract  by  BloTechnology,  Inc.  Mr.  G.  Richard  Hatterick 
served  as  the  principal  investigator  for  this  portion  of  the  study. 
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COMPUTER  BASED  MAINTENANCE  AIDS  SYSTEM: 

PRELIMINARY  DEVELOPMENT  AND  EVALUATION  OF  A  PROTOTYPE 

This  technical  paper  describes  preliminary  work  on  a  project  to  develop 
and  evaluate  a  computer  based  maintenance  aids  system.  The  system  will  be 
designed  to  store,  retrieve,  and  present  (on  a  computer  display)  data  for  use 
by  technicians  In  performing  maintenance  on  Air  Force  systems.  The  prototype 
system  and  results  of  the  evaluation  will  provide  the  basis  for  the 
development  of  a  computer  based  technical  data  presentation  system  for 
operational  use.  This  system  will  become  an  Integral  part  of  the  Automated 
Technical  Order  System  (ATOS)  being  developed  by  the  Air  Force  Logistics 
Command. 

Work  toward  developing  the  system  has  been  accomplished  under  three 
contracts.  This  paper  briefly  summarizes  the  results  of  those  studies. 
BACKGROUND 

It  has  been  recognized  for  many  years  that  conventional  Air  Force 
technical  orders  (TOs)  used  to  support  maintenance  personnel  In  the 
performance  of  maintenance  tasks  are  often  difficult  to  use.  Incomplete, 
poorly  organized,  and  written  In  terminology  that  Is  difficult  to  understand. 
Since  1962  the  Air  Force  Human  Resources  Laboratory  (AFHRL)  has  conducted 
research  programs  to  develop  improved  types  of  technical  data  for  aircraft 
maintenance.  The  emphasis  has  been  placed  upon  developing  data  which  are 
designed  for  easy  use  by  maintenance  technicians.  These  research  and 
development  efforts  have  produced  a  new  type  of  technical  data  known  as  job 
performance  aids  (JPAs). 


JPAs  present  technical  data  In  a  procedurallzed  step-by-step  format  which 
provides  detailed  Illustrations  of  referenced  components.  Emphasis  Is  placed 
on  Insuring  that  Instructions  are  complete  and  presented  In  clear,  easily 
understood  language.  In  contrast,  conventional  TOs  make  only  limited  use  of 
proceduralized  Instructions  and  provide  only  limited  Illustrations.  Emphasis 
with  conventional  TOs  Is  placed  on  "how  It  works"  more  than  on  "how  to  fix  It." 

Available  research  results  Indicate  that  the  use  of  JPAs  can  result  In 
significant  Improvements  In  maintenance  performance  and  that  JPAs  are  well 
received  by  maintenance  personnel.  One  problem  associated  with  the  use  of 
JPAs  Is  that  JPAs  require  many  more  pages  to  cover  a  system  than  do  TOs.  This 
adds  an  additional  burden  to  the  already  over-burdened  TO  system.  It  has  been 
estimated  that  conventional  TOs  require  between  300,000  and  500,000  pages  to 
describe  a  modern  complex  weapon  system.  This,  coupled  with  the  additional 
pages  necessary  for  JPAs,  results  In  a  prohibitive  amount  of  paper  and  an 
unwieldy  system.  Increasing  system  complexity,  the  consequent  necessity  for 
more  TO  pages,  and  Increasing  costs  for  technical  data  make  It  essential  that 
a  more  economical  technical  order  system  be  found. 

An  automated  job  performance/maintenance  aid  system  which  utilizes 
computer  terminals  to  store  and  present  technical  data  has  the  potential  for 
meeting  the  needs  of  field  technicians  at  lower  costs.  A  computer  based 
system  would  permit  data  to  be  presented  In  the  step-by-step  formats  required 
by  JPAs  and  would  offer  many  additional  performance  enhancement  features  not 
possible  with  a  paper  based  system.  Ready  access  to  any  TO  on  the  system 
would  eliminate  the  lengthy  search  time  now  encountered  In  locating 
Information  In  TOs.  In  addition,  the  system  would  allow  changes  and  updates 
to  TOs  to  be  done  faster  and  more  economically. 
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Design  Criteria 

An  effective  automated  job  performance/maintenance  aid  system  must  meet 
two  main  criteria:  (1)  It  must  meet  all  of  the  technicians'  needs  for 
information,  and  (2)  It  must  easily  be  used  and  acceptable  to  the  technician. 

The  maintenance  aids  system  must  provide  the  following  types  of 
information: 

1.  Procedures  for  troubleshooting  and  non-troubleshooting  tasks. 

2.  System  Information  -  descriptions,  tables  of  tolerances  and 
capacities,  theory  of  operation,  parts  information,  etc. 

3.  Diagrams  -  schematics,  wiring,  hydraulics,  etc. 

An  information  system  can  contain  all  required  information  (and  more)  and 
still  not  be  effective  if  It  is  not  used  and  accepted  by  the  technician.  For 
a  system  to  be  used  and  accepted  it  should: 

1.  Be  simple  to  use  and  require  only  limited  training  to  use  it. 

2.  Provide  for  rapid  retrieval  of  any  required  information. 

3.  Require  only  simple  skills  to  operate  (i.e.,  no  typing  or  computer 
skills  required). 

4.  Present  information  In  a  clear,  readable  format. 

5.  Provide  information  at  a  level  of  detail  appropriate  for  the  intended 
user. 

Early  Studies 

Two  studies  were  conducted  to  develop  the  basic  concepts  and  structure  of 
the  computer  based  maintenance  aids  system.  These  studies  were  conducted  by 
Behavioral  Technology  Consultants  (contract  number  F33615-77-C-0043)  and 
Behavioral  Technology  Corporation  (contract  number  F33615-78-C-0030).  Dr. 
Thomas  W.  Frazier  was  the  principal  investigator  for  both  studies.  The 
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purpose  of  these  studies  was  to  investigate  fully  the  feasibility  of 
developing  a  computer  based  job  performance  aiding  system.  A  major  objective 
was  to  develop  the  formats  and  presentation  techniques  for  the  technical  data 
with  the  maintenance  technician  in  mind.  Areas  of  investigation  included  the 
design  and  management  of:  Man-machine  dialogue  for  technician-system 
communications,  technical  data  content  In  textual  form,  and  supportive 
graphics. 

In  the  study  of  technical  data  content,  two  areas  of  concern  emerged: 

(1)  matching  technical  data  to  general  skill  levels  of  maintenance 
technicians,  and  (2)  filling  the  gaps  in  the  individuals1  repertoire  of 
skills.  The  first  area  was  addressed  by  the  concept  of  information  "tracks," 
and  the  second  by  information  "pools." 

Three  distinct  informational  tracks  were  proposed,  each  implementing  a 
different  level  of  instructional  (aiding)  detail  and  procedural ization  of 
instructions: 

Track  I  -  notes  about  technical  aspects  of  the  job,  including  tolerance 
information  and  problems  which  might  be  encountered  in  the 
course  of  the  job,  as  well  as  reminders  of  some  correct 
procedural  aspects  of  the  Job  and  special  procedures.  This 
track  was  Intended  for  use  by  highly  experienced  (skill  level 
7)  technical  maintenance  personnel.  An  example  of  the  level 
of  detail  to  be  provided  in  Track  I  data  is  presented  In 
Figure  1. 

Track  II  -  checklists  of  tasks  to  be  performed  to  complete  the  job 
presented  one  task  at  a  time,  with  access  to  pools.  This 
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T.O.  5N1-3-8-2-TS-2AT0 
AN/ASN-35  NAVIGATIONAL  COMPUTER 

BENCH  CHECKOUT  3-1 


34-60-00 

1.1842 

1 


KEY 

COMPONENT  STATUS/INDICATION  CODE 


COMPONENTS  AND  TEST  EQUIPMENT  SET  UP? . 

POWER  TURNON  AND  DISTRIBUTION: 

.NO? 

A 

-AIR  NAVIG.  MULTIPLE  INDICATOR  FLAG: 
-OFF/MAN/AUTO  SWITCH: . 

VISIBLE . 

MAN. 

.NO? 

B 

-AIR  NAVIG.  MULTIPLE  INDICATOR  FLAG: 

NOT  VISIBLE.. 

.NO? 

C 

-NAVIG.  COMPUTER  BLOWER: . 

-TEST  HARNESS,  115V  AC  SWITCH . 

OPERATING.. . . 
OFF 

.NO? 

D 

-CONTROL  INDICATOR  OFF  INDICATOR . 

-TEST  HARNESS,  115V  AC  SWITCH . 

LIT . 

ON 

.NO? 

E 

-TEST  HARNESS,  TEST  POINT  B12(H0T)... 

26  VAC . 

.NO? 

F 

-TEST  HARNESS,  TEST  POINT  E6(RETURN). 
-TEST  HARNESS,  HEADING  TRACK  READOUT 

26  VAC . 

.NO? 

G 

BETWEEN  TEST  POINTS  06  AND  D7 . PRECISE  NULL. 

•  DISTANCE  TO  GO  SENSE  AND  READOUT: 


-CONTROL  INDICATOR,  STAGE  1  DISTANCE 

TO  GO  READOUT . DECREASING _ NO?  H 

-AIR  NAVIG.  MULTIPLE  INDICATOR, 

DISTANCE  TO  GO  READOUT . DECREASING. ..  .NO?  J 

-STAGE  1  DISTANCE  TO  GO  AND 

DISTANCE  TO  GO  READOUTS . AGREE(+/-002)  .NO?  K 

-CONTROL  INDICATOR,  STAGE  SELECTOR 

SWITCH . STAGE  II  ACTIVE 

-STAGE  II  DISTANCE  TO  GO  READOUT . DECREASING. .. .NO?  L 

-AIR  NAVIG.  MULTIPLE  INDICATOR, 

DISTANCE  TO  60  READOUT . DECREASING _ NO?  M 


►  INPUT  KEY  CODE  FOR  ANY  NO  ANSWER: 

►  FOR  CONTINUATION  OF  BENCH  CHECK: 


l 


T 


Figure  1.  Example  Frame  from  Track  I  Checkout  Data* 
♦Prepared  under  contract  F33615-79-C-0021 
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track  included  only  information  about  what  tasks  are  to  be 
performed  (within  the  track),  reference  data  and  tolerances 
and  notices  of  pool  availability  for  further  aid  when 
necessary.  This  track  was  intended  for  primary  use  by 
experienced  (skill  level  5  or  3)  maintenance  personnel.  An 
example  of  the  level  of  detail  to  be  provided  in  Track  II  data 
is  presented  in  Figure  2. 

Track  III  -  complete  procedural zed  instructions  regarding  each  defined 
step  of  each  job  task,  with  all  available  data,  heavy  graphic 
support  for  parts  location  and  extensive  pool  availability  for 
informational  gap  filling.  This  track  was  intended  for  lower 
skill  level  (novice)  technical  maintenance  personnel 
performing  this  job  for  the  first  time.  An  example  of  the 
level  of  data  to  be  presented  in  Track  III  data  is  presented 
in  Figure  3. 

Extensive  opportunities  for  switching  between  tracks  were  made  available, 
so  that  technical  maintenance  personnel  receiving  more  or  less  Information 
than  was  desired  could  conveniently  move  from  one  track  to  another,  as  the 
informational  mismatch  was  discovered. 

The  concept  of  pools  refers  to  the  strategy  for  supporting  the  performance 
of  technicians  who  are  of  a  given  skill  level  but  have  gaps  in  their  knowledge 
regarding  specific  aspects  of  the  performance  of  a  particular  job.  It 
provides  a  complementary  approach  to  Information  tailoring  to  meet  a  specific 
individual's  information  support  needs.  It  is  intended  to  provide  helpful 
support,  including  embedded  training  In  prerequisite  skills  that  are  well 
established. 
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T.O.  10P3-ASQ-99-TS-1 ATO 
ASQ-99  RADAR  ANTENNA 

CHECKOUT  3-1:  ASQ-99  BENCH  CHECKOUT  (CONTINUED). 
B.  MANUAL  GROUND  SPEED  SLEW  CHECK. 

•  CHECK  MEMORY  LAMP: 

-  SET  POWER  SWITCH  TO  SLEW  AND  TERRAIN  SWITCH 
TO  LAND. 

-  DOES  MEMORY  LAMP  LIGHT  AND  REMAIN  LIT? . 


26-22-03 

2.2389 

2 

KEf 

CODE 


9  CHECK  LEFT  SLEW 

-  ON  CONTROL  PANEL  HOLD  DR  SWITCH  TO  LEFT  FOR  25 
TO  35  SECONDS,  THEN  SWITCH  TO  CENTER. 

-  CHECK  THAT  FOUR  ACTIONS  OCCUR  TOGETHER: 

1.  MEMORY  LAMP  Jj)!  LIGHTED.: . NO? 

2.  MEMORY  FLAG  J4QI  VISIBLE . NO? 

3.  ANTENNA  ROTATES  COUNTERCLOCKWISE  SMOOTHLY  AND 

THEN  STOPS  SMOOTHLY . NO? 

4.  DRIFT  ANGLE  POINTER  FOLLOWS  ANTENNA . NO? 

t  CHECK  RIGHT  SLEW. 

-  ON  CONTROL  PANEL  HOLD  DR  SWITCH  TO  RIGHT  FOR  25 
TO  35  SECONDS,  THEN  SWITCH  TO  CENTER. 

-  CHECK  THAT  FOUR  ACTIONS  OCCUR  TOGETHER: 

1.  MEMORY  LAMPJIQI  LIGHTED . NO? 

2.  MEMORY  FLAGJ1QI  VISIBLE . NO? 

3.  ANTENNA  ROTATES  CLOCKWISE  SMOOTHLY  AND  THEN 

STOPS  SMOOTHLY . NO? 

4.  DRIFT  ANGLE  POINTER  FOLLOWS  ANTENNA . NO? 


INPUT  APPROPRIATE  KEY  CODE  FOR  ANY  NO  ANSWER: 
ALL  CHECKS  OKAY?  fFQgWARM  . 


Figure  2.  Example  Frame  from  Track  II  Checkout  Procedure* 
♦Prepared  under  contract  F33615-79-C-0021 
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T.O.  12PS-CH495-TS-2ATO  29-21-03 

CH495  RADAR,  RECEIVER/TRANSMITTER  3.1131 

4 

L06IC  TREE  10-1:  HYDRAULIC  BLOWER  DOES  NOT  SPIN  WHEN 

TURNED  ON(CONTINUED) . 

•  CAN  SURGE  BE  FELT  IN  HYDRAULIC  HOSE©  WHEN  BLOWER 
SWITCH  IS  TURNED  ON  THEN  OFF? 

,  YES. 


CHECK  MOTOR  TO  BLOWER  SHAFT.  _  _ 

•  PLACE  MECHANIC'S  SIETHESCOPE  ON  OUTLET  PORT©OF  MOTOR® 

•  SET  MOTOR  SWITCH  ©TO  ON. 

•  LISTEN  FOR  A  COMBINATION  RUMBLING  AND  WHINING  NOISE  OF 
TURNING  MOTOR. 

DID  YOU  HEAR  MOTOR  NOISE? 


INPUT  EITHER 


Examples  of  types  of  pool  information  would  be  theory  of  operation  of 
equipment,  use  of  special  tools,  tolerances,  methods  and  procedures,  technical 
information,  lists  of  suitable  substitutes  for  test  equipment,  and  use  of  test 
equipment. 

The  formats  proposed  for  presenting  technical  data  in  the  three  tracks 
rely  heavily  on  graphics  materials.  This  presents  a  problem  in  the 
development  of  an  automated  technical  data  system  since  graphics  materials 
require  large  amounts  of  storage  space  and  the  storage  capacity  of  the  system 
will  be  limited.  However,  graphics  are  essential  elements  of  the  data 
presentation  formats.  Significant  reductions  in  the  number  of  graphics  is  not 
considered  to  be  a  viable  alternative.  Thus,  ways  must  be  found  to  reduce  the 
storage  required  for  each  graphic. 

There  are  a  number  of  approaches  which  may  be  taken  to  reduce  graphics 
storage  requirements.  Most  Involve  techniques  for  representing  the  graphics 
material  in  the  computer  itself.  These  approaches  primarily  involve  computer 
hardware  and  software  technology.  Oetailed  consideration  of  them  was  beyond 
the  scope  of  the  current  effort. 

Another  approach  to  reducing  storage  requirements  is  to  reduce  the 
complexity  of  the  graphics  themselves.  This  is  primarily  a  human  factors 
problem  and  was  briefly  addressed  in  this  study.  The  approach  taken  was  to 
reduce  the  complexity  of  graphics  by  elimination  of  Irrelevant  cue 
information.  Thus,  instead  of  presenting  an  illustration  which  represents  all 
of  the  features  of  the  subject  equipment  in  detail,  only  those  features  which 
are  necessary  to  permit  location  of  the  pertinent  part  of  the  equipment  are 
shown.  An  example  of  this  approach  is  shown  in  Figure  4.  Illustration  A 
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represents  a  typical  line  drawing  which  might  appear  In  a  conventional 
technical  order.  Illustrations  B  and  C  are  locator  Illustrations  which 
present  only  the  cues  necessary  to  locate  the  pertinent  parts.  A  preliminary 
evaluation  Indicates  that  this  approach  to  reducing  the  complexity  of  graphics 
and  thereby  reducing  strorage  requirements  Is  feasible.  However,  additional 
work  Is  required  to  evaluate  the  effects  of  reducing  Irrelevant  detail  and  to 
establish  ground  rules  for  developing  graphics  with  minimal  complexity. 

Third  Contract 

The  third  contract  under  this  project  was  awarded  for  the  purpose  of 
expanding  and  refining  the  concepts  developed  In  the  Frazier  studies  and 
designing,  developing,  and  evaluating  a  prototype  system.  The  contract 
(F33615-79-C-0021)  was  awarded  to  Unified  Industries,  Inc.  BloTechnology, 

Inc.  served  as  a  subcontractor  for  the  human  factors  portion  of  the  study. 

Mr.  Walt  Holmes  was  the  principal  Investigator  for  Unified  Industries,  Inc. 

Mr.  G.  Richard  Hatterlck  was  the  principal  Investigator  for  BloTechnology,  Inc. 

Work  was  Initiated  on  the  contract  In  September  1978  and  continued  until 
March  1981  when  It  was  terminated  at  the  convenience  of  the  Government. 

Contract  termination  became  necessary  when  a  change  In  requirements  for  the 
system  was  received  from  the  project  sponsor,  the  Air  Force  Logistics 
Command.  The  change  added  the  requirement  that  the  system  be  suitable  for 
deployment  In  support  of  world  wide  wartime  operations.  The  system  under 
development  was  designed  for  use  at  fixed  site  locations  and  was  not  suitable 
for  deployment.  It  was  determined  that  It  would  not  be  cost  effective  to 
modify  the  system  to  make  It  deployable.  Thus,  the  contract  was  terminated. 
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Work  was  performed  on  several  tasks  In  preparation  for  the  development  of 
the  prototype  system.  Major  areas  of  work  Included: 

1.  Identification  of  technicians'  Information  requirements. 

2.  Development  of  the  man/machine  Interface. 

3.  Identification  of  hardware  requirements. 

4.  Identification  of  software  requirements,  Including  the  design  of  a 
file  structure. 

5.  Development  of  data  presentation  formats. 

The  majority  of  the  work  accomplished  on  the  contract  was  In  the  areas  of 
the  man/machine  Interface  and  the  development  of  data  presentation  formats. 

The  emphasis  was  placed  on  developing  procedures  which  would  allow  the 
technician  to  rapidly  locate  and  retrieve  any  desired  Information  with  minimum 
effort.  A  critical  requirement  was  that  the  technician  not  need  extensive 
knowledge  of  computer  system  or  special  skills,  such  as  typing,  to  use  the 
system.  The  primary  elements  of  the  man/feachlne  Interface  are  a  special 
function  keyboard  and  associated  software.  The  layout  of  the  function 
keyboard  Is  shown  In  Figure  5.  Using  the  function  keyboard  It  Is  possible  for 
the  technician  to  accomplish  the  following  operations  with  one  or  two  key 
presses: 

Display  the  next  frame  in  sequence 

Return  to  the  previous  frame 

Display  a  list  of  available  optional  (pool)  Information 

Return  (to  a  procedural  frame)  from  a  pool  Item 

Store  or  recall  a  frame  for  later  recall 

Zoom  to  make  graphics  larger  or  smaller 


INPUT - v  / -  DISPLAY 
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Using  the  alphanumeric  keys,  the  technician  can  retrieve  a  specific  frame 
or  Item  of  Information  provided  that  he/she  knows  one  of  the  following: 

Frame  Number 

Task  Number 

Fault  Code 

MIDAS  Code 

Part  Number 

Reference  Designation 

Data  presentation  formats  were  developed  for  Tracks  II  and  III.  These 
formats  present  step-by-step  Instructions  supported  by  Illustrations  In  a 
manner  similar  to  the  job  guide  format.  The  primary  difference  between  the 
Tracks  I,  II  and  III  formats  Is  the  level  of  detail  provided.  Examples  of 
these  formats  are  presented  In  Figures  1,  2,  end  3.  It  should  be  noted  that 
the  formats  presented  are  drafts. 

Summary 

The  three  studies  described  In  this  paper  have  developed  the  basic 
concepts  and  requirements  for  an  effective  computer  based  maintenance  aiding 

system.  Requirements  developed  emphasize  the  need  to  make  the  system  easy  to 
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use  and  the  Importance  of  providing  quick  access  to  all  required  Information 
at  an  appropriate  level  of  detail.  Key  concepts  developed  or  adapted  Include: 

1.  The  three-track  system  which  provides  Instructions  at  three  levels  of 
detail.  Technicians  select  Instructions  at  a  level  of  detail  appropriate  to 
their  level  of  experience  and  training. 

2.  The  pool  concept  which  provides  for  ready  access  to  a  variety  of 
support  Information. 
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3.  A  special  function  keyboard  which  provides  for  easy  manipulation  of 
the  data  base  and  recall  of  Information  with  one  or  two  key  presses  or  entry 
of  a  short  code. 

The  next  step  In  the  program  will  be  to  apply  these  concepts  to  the 
development  of  a  deployable  computer  based  maintenance  aiding  system. 


